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HDTV TRICK PLAY STREAM DERIVATION FOR VCR 
This invention relates to the field of digital video 
recording, and in particular to the derivation, recording and 
5 reproduction of MPEG like advanced television signals at non- 
standard speeds. 

p^ryn pnTTNn OFT HF TNVRNTION 
A digital video cassette recorder employing a helical 
scanning format has been proposed by a standardization committee. 
1 0 The proposed standard specifies digital recording of standard 

definition SD television signals, for example. NTSC or PAL. and h.gh 
definition television signals having an MPEG compatible structure, 
such as a proposed Grand Alliance or GA signal. The SD recorder 
utilizes a compressed component video signal format employing intra 
1 5 field/frame DCT with adaptive quantization and vanablc length 

coding. The SD digital VCR or DVCR may digitally record either NTSC 
or PAL television signals and has sufficient data recording capability 
to record an advanced television signal. 

A specification of the GA signal is included in a draft 
20 specification documem titled Grand Alliance HDTV System 

Specification, published in the 1994 Proceeding of the 48th Annual 
Broadcast Engineering Conference Proceedings. March 20 " ^^J*- 
The GA signal employs an MPEG compatible coding method which 
utilizes an intra-frame coded picture, termed I frame, a forward 
25 predicted frame, termed a P frame and a bidirectionally predicted 

frame, termed a B frame. These tiiree types of frames occur m groups 
known as GOPs or Groups Of Pictures. The number of frames in a GOP 
is user definable but may comprise, for example. 15 frames. E^^h G^P 
contains one I frame, which may be abutted by two B frames, which 

30 are followed by a P frame. 

In an analog consumer vCK. incK rwy vw 
such as picture in forward or reverse shuttle, fast or slow motion, are 
readily achievable, since each recorded track typically contains one 
television field. Hence, reproduction at speeds other than standard. 

35 may result in tiie reproducing head, or heads, crossing mulnple tracks 
and recovering recognizable picture segments. The picture segments 
may be abutted and provide a recognizable and useful image. An 
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advanced television or MPEG like signal may comprise groups of 
pictures or GOPs. The GOP may. for example, comprise 15 frames and 
each frame may be recorded occupying multiple tracks on tape. For 
example, if 10 tracks are allocated to each frame, then a 15 frame GOP 
will comprise 150 tracks. During play speed operation. I frame data is 
recovered which enables the decoding and reconstruction of the 
predicted P and B frames. However, when a DVCR is operated at a non- 
standard reproduction speed, the replay heads transduce sections or 
segments from the multiple tracks. Unfortunately these DVCR tracks 
) no longer represent discrete records of consecutive image fields. 
Instead these segments contain data resulting mainly from predicted 
frames. However, since predicted P and B frames require preceding 
data to facilitate decoding the possibility of reconstructing any usable 
frames from the reproduced pieces of data is greatly diminished. In 
5 addition the MPEG data stream is particularly unforgiving of missing or 
garbled data. Thus to provide "Trick Play" or non-standard speed 
replay features requires that specific data be recorded, which when 
reproduced in a TP mode, is capable of image reconstruction without 
the use of adjacent or preceding frame information. The specific data. 
0 or "Trick Play" data must be semantically correct to allow MPEG 

decoding. In addition, a selection of "Trick Play" speeds, may require 
different TP data derivation and may require TP speed specific 
recorded track locations. 

To be capable of reconstruction without preceding frame 
5 data requires that "Trick Play" specific data be derived from I frames. 
The "Trick Play" specific data must be syntactically and semantically 
correct to allow decoding, for example, by a GA or MPEG compatible 
decoder. In addition tiie "Trick Play" or TP data must be inserted into 
the MPEG like data stream for recording together with tiie normal play. 
0 MPEG like signal. This sharing of the recording channel data capacity 
may impose constraints in terms of TP data bit rate which may be 
provided within the available track capacity. The TP data bit rate may 
be variously utilized or shared between spatial and or temporal 
resolution in the derived or reconstructed TP image. 
5 Reproduced "Trick Play" image quality may be determined 

by the complexity of the TP data derivation. For example, a consumer 
DVCR must derive TP data durine recordine. essentiallv in real-time 
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and with only nominal additional data processing expense added to the 
DVCR cost Thus real-time consumer DVCR "Trick Play" image quality 
may appear inferior to TP image data derived by non-real time .mage 
processing utilizing sophisticated digital image processing. With non-^ 
5 real time TP image processing for example, an edited program may be 
processed, possibly on a scene by scene basis, possibly at non-real-time 
reproduction speeds, to enable the use of sophisticated digital image 
processing techniques. Such non-real time processing may inherently 
provide higher quality "Trick Play" images than that attainable with 

10 real time processing. 

<^^i|^/^RY OF THE INVENTION; 
A method for generating an MPEG compatible digital image 
representative signal for recording to facilitate reproduction at more 
than one speed. The method comprises the steps of: receiving a digital 
15 image representative signal; temporally sub-sampling the digiUl 
image representative signal at a rate related to a trick Pl^Y speed; 
encoding the temporally sub-sampled signal to produce an MPEG 
compatible trick play signal; encoding Uie digital '^P^^"/^ 
data signal to produce a normal play MPEG compauble signal; selectmg 
20 between the trick play MPEG compatible signal and tiie normal play 
MPEG compatible signal to produce a record formatted bit stream; and. 
recording tiie record formatted bit stream. 

|>|PT? n^j^rppTTON OF THF. PR AWING: 
HGURE 1 is a simplified block diagram of an inventive 
25 system for tiie real-time generation of a "trick-play" data stream 

having low resolution. 

HGURE 2 shows a simplified block diagram of a furtiier 
inventive system for tiie real-time generation of a full resolution. 

*'trick-play" data stream. 
3 0 HGURE 3 shows a simplified block diagram illustrating an 

inventive method for generating low resolution "trick-play" data 
streams for inclusion in pre-recorded digital records. 

FIGURE 4 shows a simplified block diagram illustrating a 
further inventive metiiod for generating "trick-play" data streams use 
35 for inclusion in pre-recorded digital records. 

HGURE 5 illustrates the derivation of predicted 

macroblock DC coefficients. 
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FIGURE 6 shows a simplified partial block diagram 
illustrating a further inventive method for non-real-time generation 
of pre-recorded records. 

FIGURE 7 shows a simplified partial block diagram 
5 illustrating another inventive method for non-real-time generation of 
pre-recorded records. 

PETAB.ED PES CR TO Q N : 
In a consumer digital video cassette recorder major 
considerations in the real-time generation of a trick-play stream are 

10 the complexity and cost of processing required, and the need to keep 
this cost at a reasonable level. For this reason, the processing utilized 
in the generation of a real-time trick-play data stream may be 
limited to extracting pieces of the existing bit stream and 
implementing minor modifications to bit-stream parameters. 

1 5 "Trick-play** data streams must be produced in real-time 

by extracting independent intra-information pieces from the original 
data stream. This intra-information may come from intra-frames, 
intra-slices, and/or intra-macroblocks. The source selected for I 
frame data derivation depends on the form of intra refresh employed 

20 in the original stream, and for exemplary purposes it is assumed that 
either intra-frame or intra-slice refresh method is employed. 

In a first inventive method of real-time generation, a low 
spatial resolution "Trick Play" data stream is derived. The low spatial 
resolution trick-play stream may, for example, have resolution 

25 according to the CCIR 601 standard, (720 x 480 pixels), regardless of 
the original HDTV stream resolution. Since the effective available bit- 
rate for trick-play streams is limited to nominally 2 M. bits/sec, 
employing low spatial resolution in this manner results in fewer bits 
being used per frame, and thus a relatively high temporal resolution 

30 may be achieved. However, this low spatial resolution may only be 
practical if an advanced television decoder and display is capable of 
such resolution. 

In a second inventive method a trick-play stream is 
generated having the same resolution, or pixel count, as the original 

35 HDTV material. However, since the usable trick-play bit-rate is 

limited by the recording channel capacity of nominally 2 M. bits/sec, 
a trade-off exists between spatial and temporal resolution. Thus the 
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provision of a full spatial resolution "Trick Play" mode effectively 
requires that the temporal resolution be reduced to remain 
commensurate with the TP data channel capacity. 

The first inventive method for real-time generation of a 

5 low spatial resolution "Trick Play" data is illustrated in ™URE 1. In 
this exemplary block diagram, trick-play speeds of 5x, ^^x and 35x 
are generated. For each TP speed, low-resolution. mtra-coded frames 
are constructed from a received MPEG like transport stream. By 
detecting MPEG header information in the transport stream down to 

1 0 the sUce level, intra slices can be extracted, processed and used to 
create a single I-frame in memory 110. The extraction and 
processing stage 100 performs three tasks; extracting macroblocks for 
the construction of a TP I-frame. re-encoding DC transform 
coefficients when necessary using DPCM encoding, and discarding 

15 unwanted AC transform coefficients when necessary. Havmg 

constructed and stored a low-resolution TP I-frame m memory 110. it 
is utilized in tiie generation of speed specific data streams for each 

trick-play speed. • o« 

A radio frequency carrier, modulated responsive to an 
20 MPEG compatible signal, is received by receiver 05. The modulated 
^er may be sourced from either an antenna or a cable, not shown. 
Receiver 05 demodulates and processes tiie received earner to 
produce an MPEG compatible advanced television transport stream 

09. 

, 5 ' The advanced television transport stream 09, is 

demuWplexed in block 20 to oburin only the ^^"^^'^'^ ^^'"^"^^^ 
S,«am or PES stream corresponding to the advanced "'""7 J^" 
information. The PES stream is decoded in block 30 to extract from 
the packets, the MPEG encoded video stream payload. Havmg 

30 extracted the MPEG encoded stream, the required ^''"^-^^^ 

information may be detected and extracted. Sequence det«t.on block 
40 examines the bit stream for the occurrence of a sta« code 
characterized by twenty f.ve O-s followed by 1. followed by an 8 bu 
address indicating MPEG video header. Picture detecuon .s 

35 performed in block 50 and in block 60 slice layers are detected^ 
Since an intra coded "trick-play" I frame is to be constructed only 
intra-slices are extracted. Intra-slices contain only mtra-coded 
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macroblocks. and are characterized by a 1-bit intra_slice flag in the 
slice header. Thus when the intra_slice flag is set to 1 the entire slice 
is passed to the "data extraction and processing" stage 100. The intra 
detection process of block 70 assumes that either intra-frame or 
5 intra-slice refresh techniques are employed and also that the intra- 
slice flag in the slice header is set when appropriate. If the 
intra_slice flag is not set or intra-macroblock refresh is used then a 
further level of detection down to macroblock level is required. 

The data extraction and processing stage 100 selects from 
10 the intra-coded macroblocks extracted in block 70, only intra 

information which is utilized for constructing various trick-play data 
streams. In addition block 100 performs any processing which may 
be necessary to ensure the syntactic and semantic correctness for 
MPEG compatibility of tiie resulting reconstructed TP I-frame. Since 
15 the reconstructed TP I-frame is of lower spatial resolution Uian the 
original MPEG stream, only a sub-set of tiie detected intra- 
macroblocks is required. To determine which macroblocks or MBs are 
to be kept and which are to be discarded, either a mathematical 
function or a predefined look-up table may be employed. The 
20 resulting lower spatial resolution frame results from the selected 
patchwork of macroblocks. A controller stage 90 is coupled to 
processing stage 100 and provides either, calculation required by tiie 
matiiematical function or provides the look up table for determining 

macroblock selection. 

25 The relationship between the MB position in tiie new low- 

resolution I-frame, 

(mb(i. j). i=0, 1, 2, ... n-1, j = 0. 1, 2, ... m-1, where m and n are 
the new I-frame width and height in MBs respectively and i and j 
refer to the MB row and column) and the original full-resolution 

3 0 frame ((MB(I, J), 1=0, 1, 2. ... N-1, J=0, 1, 2. ... M-1, where M and N are 
Uie original frame widtii and height and I and J are the MB row and 
column), tiie relationship is given by: 

i (low-resolution row) = [I.(n-1)/(N-1)] 

3 5 j (low-resolution column) = [J.(m-l)/(M-l)l 
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where the product of the square brackets [x] denotes the integer 

value closest to x. 

The low resolution TP I frame utilizes a subset of the 
macroblocks from the original frame with the remaining non-selected 
5 MBs being discarded. FIGURE 5 iUustrates an exemplary 4:2:0 

sampled signal comprising three intra-coded macroblocks MBl, MB2 
and MB3. where each comprises blocks 0. 1. 2. 3. 4 and 5. Macroblock 
2 is crossed through to illustrate non-use in constructing the reduced 
resolution TP I frame. The DC coefficients of each luminance and 

1 0 chrominance block are depicted in HGURE 5 with dark stripes. The 
DC coefficients are predicted from within each macroblock, with the 
DC coefficient of the first block of an MB being predicted from the last 
DC coefficient of the immediately preceding MB of the slice. The 
arrows in FIGURE 5 illustrate the prediction sequence. Thus, if the 

1 5 preceding MB, for example, MB 2 of HGURE 5 is not selected, certain 
DC coefficients must be re-calculated from the newly abutted 
macroblock, as depicted by arrows NEW of FIGURE 5, and re-encoded 
using DPCM. This re-encoding process is performed as the 
macroblocks are written to the I-frame memory 110. 

20 If the HDTV video sequence originated from an interlaced 

scanning source, an optional processing step may be included to 
remove interlace "flicker" exhibited by frozen interfaced fields 
containing motion. If the temporal resolution of the reconstructed 
trick-play stream is such that the same frame (two fields) is 

25 displayed for more than one frame period, then such interfaced 
"flicker" may be very noticeable. In field-coded macroblocks this 
"flicker" artifact may be eliminated by copying the top two blocks of 
the macroblock, blocks 0 and 1, to the lower two blocks, blocks 2 and 
3. This copying within the macroblock effectively makes both fields 

3 0 the same thus removing any field-to-field motion from the frame. 

This re-encoding process is performed as the macroblocks are written 

to the I-frame memory 110. 

A further function performed by processing stage 100 is 
the removal of AC coefficients from each macroblock which cannot be 
3 5 accommodated in the newly constructed TP I-frame due to the low 
bit-rate available for the trick-play streams. To accomplish this, each 
Wnrk is variahle-length-decoded to the ooint where the block will be 
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padded with zeros, indicating the last coefficient of that block. The 
number of bits for each block are stored and accumulate in a buffer. 
The bits arc counted and when a count exceeds a predetermined 
number the remaining AC coefficients are unused or deleted. The 

5 number of bits per TP MB depends on the overall rate allowed for 
each trick-play stream and the temporal resolution or number of 
frame updates per second. 

The block diagram of FIGURE 1 illustrates the formation of 
trick-play data streams having the same allocated bit-rate. If the 

10 rate differs significanUy between TP speeds, for example, to provide 
differing resolution at each speed, tiien the number of AC coefficients 
retained in I-frame memory 1 10 will also differ for each speed. 
Hence I-frame memory 110 cannot be shared and separate I-frame 
memories may be required for each TP speed or bit rate. 

15 The inventive low-resolution TP I-frame assembled in I- 

frame memory 110 is coupled to tfiree trick-play stream generation 
stages: 5 times, block 145; 18 times, block 160 and 35 times block 
170. In exemplary FIGURE 1. each trick-play stream may be 
allocated the same bit-rate and temporal resolution, which may 

20 represent a preferred configuration. However, not every 

reconstructed TP I-frame is used for each TP speed. For example, if 
the I-frame refresh rate in die original stream is once every fifteen 
frames (M=15) and tiie temporal resolution used by each trick-play 
stream is selected to be tiiree, i.e. the number of frame times between 

25 frame updates, then for 5 times speed; 

(5x speed). (3 frame repeats)/(15 frame refresh) = 1.0 

tiius every TP I-frame will be used. Similarly for 18x and 35x 
3 0 speeds, 

(18).(3)/(15) = 3.6 
(35).(3)/(15) = 7.0 

3 5 Thus at 18x speed approximately every third or fourth I-frame is 
used, and at 35x speed every seventh I-frame is used. If it is 
assumed that the intra-refresh period in an advanced television 
stream is 0.5 seconds (M=15 for 30 fps source) then a three-frame 
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holding time for 5x speed is the highest possible TP temporal 
resolution. For simplicity and consistency a three-frame holding time 
may be used for the remaining TP speeds. A higher temporal 
resolution of two-frames or single-frame holding time could be used 
5 for higher TP speeds since lower temporal resolution at higher speeds 
may give a false sense of slower than actual trick-play speed. 
Assuming that the effective trick-play bit-rate is constant, the 
provision of a higher temporal resolution would consequently require 
a lower spatial resolution quality. 
10 The reconstructed TP I-frame is read from memory 110 

and packaged, according to TP speed, by blocks 145. 160 and 170 
which add the appropriate MPEG picture headers and a PES layer. 
The advanced television transport stream 09 is buffered by buffer 
15. which generates signal 10. a transport stream for normal play 
1 5 sp^ed processing. Normal play transport stream 10 is coupled to 

multiplexor MUX 150. Multiplexor MUX 150 is controlled responsive 
to recorder 210 servo signals to generate an output bit stream having 
a sequence which when recorded produces a predetermined track 
format. The recorded track format is selected to provide the desired 
20 recorded TP bit rate and to facilitate specific physical location of 
speed specific TP I-frame packets within specific recorded tracks. 
The recorded track format thus facilitates replay at normal speed and 
at the predetermined trick-play speeds. The TP I-frame packets. 5x 
signal 121. 18x signal 131 and 35x signal 141. are coupled to 
25 multiplexor MUX 150 which inserts the I-frame packets for each TP 
speed into the normal play transport sueam. Thus a valid, MPEG like, 
transport stream is formatted for record processing by recorder 210 

and recording on tape 220. 

To minimize TP bit rate, in place of repeated TP I frames, 

30 frame repeats or holding times, may be implemented by writing 

empty P-frames between I frames in the video stream. An empty P- 
frame results in the decoder predicting from the previous frame, i.e. 
the TP I frame. Alternatively, frame repeats may be implemented by 
setting the DSM_trick.modc_flag in the PES layer and calculating the 

35 Presentation Time Stamp and Decode Time Stamp PTS/DTS values 

such that each TP I frame is presented the necessary number of frame 
tim^c ,n«rt FJthPr framP rP.npat mpfhort nroduces the same resull. 
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However, the second method requires no extra processing of the TP 
stream on playback and hence, adds no extra cost to the unit. 
However, the second method requires that the optional 
DSM_trick_mode_flag is supported in advanced television decoders. 
5 With this second method, the extra processing is implemented in the 
advanced television decoder. Either frame repeat method may 
implemented during speed speciHc stream generation in blocks 145, 
160 and 170. 

The inventive trick-play stream generation techniques 
10 described above were employed to produce trick-play speeds of 5x, 
18x and 35x with a spatial resolution of 720 x 480 pixels, and an 
effective trick-play data rate of 2.0 Mbps. The various trick-play 
speeds were evaluated and may be summarized by the following 
points: 

15 Data for each trick-play speed was generated representing 

independent low-resolution (720 x 480 pixels). MPEG compatible 
transport streams. 

Each TP stream contains only intra-coded frames thus allowing 
the same trick-play stream to be used for both Fast Forward and Fast 
20 Reverse TP modes. 

To retain a 16:9 aspect ratio, the actual spatial image size is 
sampled to 720 x 384 pixels, with the remaining area above and 
below the TP image black. 

The temporal resolution is such that a constant three-frame 
25 holding time is used resulting in an effective rate of 10 frames per 
second. 

Each I frame of the trick-play streams comprises a selection of 
sampled macroblocks from the original stream. The bit rate of 2.0 M. 
bits/sec. and three-frame holding time allows most AC coefficients to 
30 remain in the selected macroblocks for typical test material. 

The overall subjective spatial resolution is fair, being dependent 
on the amount of motion and image complexity in the source material. 
A picture rate of 10 fps provides good temporal resolution. The trick- 
play data stream may be decoded to produce recognizable trick-play 
3 5 video images and hence is acceptable for tape search usage. 

The inventive low-resolution real-time trick-play mode 
oreviouslv discussed oroduces recoenizable snatial imaees at a 
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relatively high temporal resolution. However, as already mentioned, 
this mode may be used if an advanced television receiver/decoder 
unit is operable at lower resolution, for example, such as that 
produced by CCIR recommendation 601, However, if operation at a 
5 lower resolution is not provided, then trick-play data must be 

derived having nominally the same spatial resolution, i.e. the same 
pixel count as the original source. HGURE 2 illustrates an inventive 
exemplary system for generating full-resolution, real-time tirick-play 
streams. Three trick-play speeds of 5 times. 18 times and 35 times 
10 are illustrated. The difference between the full-resolurio^^^^^ 

HGURE 2 and the low-resolution scheme illustrated in HGURE 1. is m 
data extraction and processing block 105. and stream generation 

blocks 155. 165 and 175. 

The transport stream decoding and intra detection 
1 5 depicted in blocks 20. 30. 40. 50. 60. and 70 operate and function as 
described for the low resolution TP system of HGURE 1. As descnbed 
for tiie low resolution TP system, the purpose of the data extraction 
and processing stage, block 105. is to extiact only intra information 
which is required for forming trick-play streams and to perform any 
20 processing which is required to guarantee Uie syntactic and semantic 
correctness of the resulting TP I-frame. The functionality of block 
105 differs from that of block 100 in that Uie regenerated I-fwme 
must have die same resolution, or pixel count, as the original data 
stream. Hence, all intra macroblocks are used to reconstruct tiie new 
25 TP I-frame. Since no MBs are deleted, no re-encoding of DC transform 

coefficients is required. 

The major funcrion of processing block 105 is Ihe removal 
of AC coefficients from each macroblock which, as a consequence of 
the trick-play bit-rate cannot be accommodated in the new TP l- 

30 fr»ne. The low TP channel bit-rate, nominally 2 M. bit/sec. forces a 
trade-off between the number of AC coefficients used. t.e. spattal 
resolution, and the temporal resolution, or frame update rate of Ae 
trick-play stream. This spatial versus temporal trade-off was also 
present in the derivation of the low-resolution stream. However .n a 

35 ^H o tion frame, i.e. same pixel count, the DC coefficienu alone 
are likely to represent more bits than all the coeffic.ents. both AC and 
DC assembled in a. low-resolution TP frame. Thus any hmtted 
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inclusion of even a few AC coefHcients in each full-resolution 
macroblock will produce a significant reduction in the temporal 
resolution, i.e. the frame update time will be lengthened, with more 
frame repeats. Thus to facilitate constant temporal resolution in full- 
5 resolution trick-play streams, a system may employ only the DC 
coefficients of each macroblock with all AC coefficients being 
discarded. In addition, discarding the AC coefficients reduces 
processing complexity since only variable-length decoding of the 
DPCM value of the DC coefficient is required. FIGURE 2 illustrates an 

1 0 exemplary system where each trick-play speed has the same bit rate, 
and thus the same I-frame memory may be shared between the 
three TP speeds. 

As discussed previously, if the original HDTV video images 
were generated by interlaced scanning, then an optional processing 

1 5 step may be included to remove interlace "flicker" exhibited by 

frozen flelds containing motion. One such method has already been 
described. However, since this exemplary high resolution TP system 
uses only DC transform coefHcients, a simpler and more efflcient 
method may be provided by setting the frame_pred_frame_dct flag 

20 in the picture_coding_extension section to T. This flag indicates that 
all MBs were frame encoded, thus a previously field-coded block, 
which could produce 'flicker', is decoded as a frame-coded block. The 
result is that each Held is placed in either the upper or lower portion 
of a block and any 'flicker' is removed. This method of flicker 

25 elimination also reduces the number of bits used in the 

macroblock_modes section since the dct.type flag can no longer be 
present if frame_pred_frame_dct_ is set to '1'. 

The reconstructed TP I-frame is assembled in memory 
115, and coupled to three trick -play stream generation stages, 5 

30 times speed depicted in block 155, 18 times speed in block 165 and 
35 times speed in block 175. The exemplary system of FIGURE 2 
assumes that each trick-play stream has the same effective bit-rate 
and hence the same approximate temporal resolution. As discussed 
previously, not every reconstructed TP I-frame is used for each 

35 speed. However TP I-frame utilization may be further limited for the 
following reason. Although each TP I-frame has the same number of 
coefficients, for example. DC onlv. each TP I-frame mav not have the 
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same number of bits since the DC coefficients are variable length 
encoded. Therefore, a constant temporal resolution or frame holding 
time, cannot be fixed for each trick-play stream. Instead the frame 
holding time will vary slightly over time with the number of bits 
5 required to encode or form each TP I-frame. For each trick-play 
speed, the respective "stream generation" stages, 155, 165 and 175, 
wait until enough bits have been accumulated in buffer 105 to encode 
a TP I-frame. Then if the TP I-frame accumulated in the buffer at 
the time is a new TP I-frame, i.e. one which has not yet been encoded 

10 in the specific trick-play speed, the TP I-frame is encoded and the 
number of bits used will be subtracted from those available. If every 
I-frame was the same size and each trick play speed was allocated 
the same effective bit-rate, this scheme would be equivalent to that 
described for the low-resolution system and the frame refresh period 

15 would be constant for all speeds. The reconstructed TP I-frames are 
read from memory 115 and packaged by stream generators 155, 165 
and 175 to form a MPEG compatible transport streams in exactly the 
same way as detailed for the low-resolution system. 

The inventive full spatial resolution trick-play stream 

20 generation technique described above was evaluated at an effective 
trick-play data rate of 2.0 Mbps, for trick-play speeds of 5x, 18x and 
35x. The performance may be summarized as follows: 

An independent, TP I-frame-only MPEG compatible transport 
stream may be recorded for each trick-play speed. 

25 The temporal resolution varies with scene complexity and is 

lower, having longer frame holding times than the low spatial 
resolution trick-play system previously described. The average and 
the variation in holding times experienced for typical source material 
are shown in the following table: 



TP SPEED 1 AVERAGE HOLDING 
1 TIME IN FRAMES 


VARIATION IN | 
FRAMES 


5X 


5 FRAMES 


5 - 8 FRAMES 


18X 


5 FRAMES 


5 - 8 FRAMES 


35X 


5 FRAMES 


5 - 8 FRAMES 
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Note: Because an identical effective trick-play bit-rate is used 
for all speeds, the temporal resolution will always be similar (if not 
identical) for each speed. 

Each TP I-frame uses only DC coefficients. 

5 The overall quality of spatial resolution is only fair since only 

DC coefficients are used. The quality of temporal resolution may vary 
between poor and fair, depending on the level of complexity within 
the TP encoded material. However, the resulting trick-play images 
are recognizable and acceptable for tape search usage. 

10 The major differences between real-time trick-play and 

pre-recorded trick-play data stream derivation, result from the 
constraints of cost and lack of complexity imposed in a consumer 
recorder/player. The consumer unit must derive and record the 
trick-play data stream while recording normal replay data, i.e. the 

15 trick-play data stream is derived in real-time. With pre-recorded 
material, trick-play data streams may be derived directly from an 
original picture source rather than from a compressed MPEG encoded 
stream. Speed specific TP data streams may be derived 
independently of one another and independently from the actual 

20 recording event. Thus pre-recorded trick-play data may be derived 
in non-real time, possibly at non-standard or slower frame repetition 
rates. Since the constraints of the consumer real-time method no 
longer apply, the quality of trick-play reproduction achieved by pre- 
recorded material may be significantly higher. 

25 A first inventive method of pre-recorded TP data 

derivation provides a spatial resolution of for example, CCIR Rec. 601 
having a resolution of 720 x 480 pixels, regardless of the original 
HDTV stream resolution. A second inventive method constructs a 
trick-play stream of the same resolution, i.e. pixel count, as the 

30 original HDTV material. 

FIGURE 3 illustrates an exemplary block diagram showing 
an inventive method for generating low-resolution, pre-recorded 
trick-play data streams. Regardless of the format of the original 
HDTV video material 09, temporal processing block 30, performs 

3 5 temporally subsampling which produces a 30 Hz, progressive signal 
31. The operation of this stage may differ depending on whether the 
original source material is nropressivP with ^ SQ Q4/fin Hz fr^me. tmp 
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or interlaced with a 29.97/30 Hz frame rate. With progressively 
scanned source material, the frame rate may be reduced by droppmg 
every second frame from the sequence. By dropping alternate frames 
a progressive sequence results having half the temporal resolution of 
5 the original source material. With interlaced source material, the 
frame rate remains the same but only one field from each frame is 
used. This processing results in a progressive sequence of half the 
vertical resolution and the same frame rate. 

The progressively scanned frames, signal 31 is coupled to 
1 0 block 40, which generates a lower resolution signal having, for 
example, the resolution delivered by CCIR Rec. 601. Each 
Progressively scanned frame is resampled to 720 x 384 pixels to 
retain the 16:9 aspect ratio, and padded with black upper and lower 
borders to produce a letter-box' format of 720 x 480 pixels. 

The HDTV signal is now represented by signal 41. having 
a lower spatial resolution of 720 x 480 pixels, progressively scanned 
with a 30 Hz frame rate. Signal 41 is coupled to blocks 50. 60. 70 
which implement speed-dependent temporal subsampling. Each 
trick-play stream is constructed to have the same temporal resolution 
20 or frame holding time of 2 frames, i.e. every frame will be repeated 
once. Therefore, at N times trick-play speed, the frame rate is 
reduced from 30 Hz to 30/2N Hz. Thus, the resulting recorded frame 
rates are as follows. 5x becomes 30/10 Hz. 18x becomes 30/36 Hz 
and 35x becomes 30/70 Hz. Since every frame is presented twice 
25 and the display rate is 30 Hz. the effective speed of scene content 
remains correct at each TP speed. 

The temporal subsampling blocks 50. 60, 70. generate 
output bit streams 51, 61 and 71 respectively, which are coupled to 
respective MPEG encoders 120, 130 and 140 to format MPEG 
30 compatible bit streams. Since the MPEG compatible encoding is the 
same for each speed, and because in a prc-recording environment 
real-time processing is not necessary, the same MPEG encoding 
hardware may be used to encode the normal-play stream and each 
trick-play stream. This commonalty of usage is indicated by the 
3 5 broken line enclosing the MPEG encoder blocks 100, 120 130, and 

140 The temporally subsampled bit streams 51, 61 and 71 are MKtu 
.n.nH.H .c l-framps Each 1-frame is repeated once by employing 
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the DSM_trick_play_flag, located in the PES layer as described 
previously. The resulting MPEG compatible streams representing 
normal play speed NP, stream 101, and trick-play speeds of 5x, 
stream 121, 18x, stream 131 and 35x, stream 141, are coupled for 
5 record formatting by multiplexor 150. Multiplexor 150 effectively 
selects between the various MPEG streams to generate a sync block 
format signal 200. suitable for record processing by record replay 
system 210 and writing to tape 220. As described earlier, the use of 
predetermined TP speeds allows speed specific TP data to be 

10 positioned, or recorded, at specific sync block locations within 

recorded tracks. Thus multiplexor 150 formats sync block signal 200 
to locate speed specific TP 1 frame data at specific sync block 
locations within the recorded tracks. These specific locations facilitate 
reproduction at the various specific TP speeds. 

15 FIGURE 6 is a partial block diagram illustrating a further 

inventive arrangement of the non-real-time "trick-play" apparatus of 
FIGURE 3. Speed specifically processed TP signals 51, 61 and 71 are 
coupled to memories 520, 530 and 540 which store the 5 times, 18 
times and 35 times processed digital image signals respectively. The 

20 storage memories may be provided by means of disk or magnetic 

tape recording and retrieval systems. The original HDTV signal 09 is 
also stored in memory 500. Production of the prerecorded media or 
tape is facilitated by the sequential selection between the various 
stored digital signal sources to form an output signal which is MPEG 

25 encoded by encoder 100 and recorded on the media. A multiplexor 
150 is controlled to select between the various digiul signal sources 
to form an output signal for MPEG encoding. The MPEG encoded 
signal 200 has the various signal components arranged such that a 
recording may be replayed at normal and trick play speeds. Thus the 

3 0 inventive arrangement of FIGURE 6 facilitates the non-real-time, and 
independent derivation of both normal play and trick play digital 
signal sources for encoding as MPEG compatible bit streams. 

A further inventive arrangement is illustrated in block 
600 which shows an alternative use for the multiplexed normal and 

3 5 trick play MPEG encoded data stream produced by MUX 150. The 
alternative system of block 600 replaces MPEG encoded data stream 
200, recorder 210, format control signal FMT CTL, decoder 07 and 
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display 300. In block 600. an MPEG encoded data stream ZM is 
tLunicated to a tnutsmitte, 400 for coupling to 0-d« 07 a^ 
display 300. A user viewing display 300 may chose to 
material being viewed and initiates a remote ""'"'""'rf 

i CTRL which is communicated to multiplexor 150. Multiplexor 150 
responds to U.e users remote command and selects, for e"-"?'*- 
5 Ls speed bit stream 521 to be coupled for N^EG -odmg »d 
subsequent transmission, decoding and display. S.mtlarly the user 
,„ay chose to view in Uie reverse direction at. for example. 5 times 

0 IL. This selection may be faciliutted by reading the 5 umes ^y 
sjeed memory 520 in reverse. Thus ti,e 

provides user controlled selection between normal and tnck play 

MPEG encoded data streams. 

FIGURE 7 is a partial block diagram illustrating another 
5 inventive arrangement of tiie non-real-time "trick-play" apparatus of 
™3. to FIGURE 7 boti, normal play «.d trick play pro«^ 
digital signals 09. 51. 61 and 71 are coupled for encoding as MPEG 
compatib^ bit streams by encoder 100. WiU. non-reaHim^ s gnal 
processing and pre-recorded material preparation, signals 09 51, 61 
,0 ^71 m'ay be derived separately and individually "''P'f 

encoding by a single encoder 100. THe individually "^^^ 
streams 101. 121. 131 and 141 ate stored ■» ""T™! 
and 580 representing normal play and 5x. 18x and 35x bit stream 
^iectively THe storage memories may be provided by m«ns of 
» disfor magnetic tape recording and retrieva systems^ Mem«.« 
550 560 570 and 580 produce output signals 501. 521 531 ana an 
whth^ coupled to multiplexor 150 which '-<'-«;>«'j;^-'" 
to recorder 210 to generate an MPEG compatible record bit stream 
TorT^lS such as to provide reproduction at normal play speed and 
30 at the predetermined "trick-play" speeds. 

A further inventive arrangement is illustrated m b ock 
600 which shows an alternative use for the multipl«ed .^^»d 
trick Play MPEG encoded dau stream produced by MUX 150. Tne 
"ll'e system of «ock ^"^1 ^ 

3 5 200. recorder 210, format control signal FMT CTL, 

disDlav 300 m block 600. an MPEG encoded data stream 202 is 
Zln c^ed lo a transmitter 400 for coupling to decoder 0„7 »d 
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display 300. A user viewing display 300 may chose to advance the 
material being viewed and initiates a remote control command REM 
CTRL which is communicated to multiplexor 150. Multiplexor ISO 
responds to the users remote command and selects, for example, the 
5 S times speed bit stream 521 to be coupled for MPEG encoding and 
subsequent transmission, decoding and display. Similarly the user 
may chose to view in the reverse direction at, for example, 5 times 
speed. This selection may be facilitated by reading the 5 times play 
speed memory 520 in reverse. Thus the arrangement of block 600 

10 provides user controlled selection between normal and trick play 
MPEG encoded data streams. 

The exemplary, low spatial resolution TP system 
illustrated in FIGURE 3, and described above, produces trick-play 
quality significantly higher than that attainable from real-time 

1 5 derived trick-play streams. The results produced may be 
summarized as follows. 

During recording, an independent, I-frame only, low-resolution 
(720 X 480 pixel) MPEG compatible stream is written to tape for each 
trick-play speed. 

20 The actual spatial image size is 720 x 384 pixels, to retain 16:9 

aspect ratio, presented in a "letter box" format. 

The temporal resolution is effectively 15 frames/second for 

each trick-play speed and produces good to excellent quality which 

remains constant for each speed. 
25 The spatial resolution produced by a 2.0 Mbps data rate and 

720 x 480 pixels resolution is good to very good, depending on the 

complexity of the source material. 

Overall, the trick-play image quality exhibited with this scheme 

is very high. 

30 The low-resolution pre-recorded trick-play system shown 

in FIGURE 3 and describe above produces good quality spatial images 
at a relatively high temporal resolution. However, such a low- 
resolution method may be used providing the advanced television 
decoder/receiver unit is able support the lower resolution display 

3 5 format. 

FIGURE 4 is an exemplary block diagram of an inventive 
full-resolution, pre-recorded trick-olav stream eeneration svstem. 
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providing trick-play speeds of, 5x. 18x and 35x. As previously 
discussed, pre-recorded trick play data stream derivation ^^y^^ 
generated from the original, uncompressed, source matenal. FIGURt 
4 illustrates the generation of normal-play and trick-play bit streams. 
5 however these may be generated independently of one another, 
directly from the HDTV source material. Since this system provides 
full-resolution, no spatial sub-sampling is required and hence less 
processing is required than that shown in HGURE 3. Since the 
original, uncompressed, source material may be used, frames wh.ch 
10 are to be intra-coded may be chosen exactly to suit the tnck play 
speed, rather than selecting I frames from an encoded stream. In 
addition a constant temporal refresh rate can be maintained, which is 

more pleasing to the user. 

The original HDTV video signal 09 is shown coupled to 

1 5 MPEG encoder 100 which generates an MPEG stream 101 for normal 
play speed operation. Signal 09 is also coupled for temporal 
subsampling in blocks 55. 65 and 75 respectively. For a tnck-play 
speed of N times, only every Nth source frame may be utilized for 
coding. However, depending on a desired trade-off between spatial 

20 and temporal resolution, the actual frames used for encoding may be 
closer to every 5Nth or 8Nih frame in order to provide an acceptable 
spatial resolution. Hence frame holding times, or temporal resolunon, 
are similar to those of the real-time, full-resolution system described 

earlier, ^ . ^ . ^ 

25 Having selected a frame holding or update nme. for 

example, every 5Nth frame for each N rimes trick-play 'P^^l^he 
HDTV stream, signal 09. is temporally sub-sampled for each TP speed. 
The 5 times TP stream is derived in block 55 which temporally sub- 
samples by a factor of 1/5N. or 1/25. i.e. 1 frame in 25 ts selected to 
30 genLe output signal 56. Similarly, the 18 times TP streatn .s 
derived in block 65, which temporally sub-samples by a factor of 
1/5N, or 1/90 and generates output signal 66. The 35 times ir 
stream is derived in block 75, which temporally ''1' » 

factor of 1/5N. or 1/175 and generates output s.gnal 76. The three 
3 5 sub-sampled TP bit stream signals. 56, 66 and 76 are coupled or 
MPEG encoding in encoder blocks 120. 130 and 140 respect.vely. 
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Since MPEG compatible encoding is the same for each 
speed and because rcal-ume processing is not necessary in a pre- 
recording environment, the same MPEG encoding hardware may be 
used to encode the normal-play stream and each trick-play stream. 
This commonalty of usage is indicated by the broken line enclosing 
the MPEG encoder blocks 100. 120. 130. and 140. The temporally 
subsampled bit streams 56. 66 and 76 are MPEG encoded as I-frames. 
Because the frame update time is constant throughout each trick-play 
stream, so is the number of bits allocated for each I-firame. The 
0 frame holding times, or I-frame repeats may implemented by 
employing the DSM_trick_play_flag as previously described. The 
resulting MPEG transport streams representing normal play speed NP, 
stream 101. and trick-play speeds of 5x. stream 121. 18x. stream 131 
and 35x. stream 141, are coupled for record formatting by 
5 multiplexor 150. Multiplexor 150 effectively selects between the 
various MPEG streams to generate a sync block format signal 200. 
suitable for record processing by record replay system 210 and 
writing to tape 220. As previously described, predetermined TP 
speeds allow speed specific TP data to be positioned, or recorded, at 
to specific locations within recorded tracks. Thus multiplexor 150 

formats sync block signal 200 to locate speed specific TP I frame data 
at specific sync block locations which facilitate reproduction at the 

various specific TP speeds. 

The inventive arrangements of HGURES 6 and 7 may also 
25 be applied the non-real-time "trick-play" generation arrangement of 
HGURE 4. As has been described, the arrangements of HGURES 6 and 
7 may facilitate non-real-time, and independent derivation of normal 
play and trick play digital signals, and the encoding of MPEG 
compatible bit streams. 
30 The storage and retrieval memories 550. 560. 570 and 

580 of HGURE 7. generate MPEG compatible output signals 501. 521 
531 and 541 which are illustrated coupled via multiplexor MUX 150. 
The the MPEG output stream 200 generated by MUX 150 may. in an 
alternative inventive embodiment, be coupled to a transmission 
3 5 system for distribution for user display. Multiplexor MUX 150 may 
be controlled to select between memory derived MPEG compatible 
output signals responsive to user commands. For example, a user 
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Viewing a program at normal speed will receive and decode bit 
stream signal 202. The user may desire to advance or fast forward, 
the program by remotely selecting viewing of for example the five 
time speed MPEG bit stream. The user remote command for 5x 
5 replay, causes multiplexor 150 to select MPEG bit stream 521 which is 

output to the user. 

The constraints of retaining full spatial and temporal 
resolution, result in a trick-play quality which is very similar to that 
achieved by the full-resolution real-time method. However, this pre- 
0 recording method has an advantage that the frame holdmg time is 
constant. The trick-play stream generation technique described 
provides trick-play speeds of 5x, 18x and 35x, having full spatial 
resolution, and an effective trick-play bit rate of 2.0 Mbps. The 
performance may be summarized as follows: 
5 During recording, an independent, I-frame only, MPEG stream is 

written to tape for each trick-play speed. 

The spatial resolution is the same as tiie source matenal. 
The temporal resolution is fixed having a 5 frame holdmg time. 
Each I-frame uses all DC and some AC coefficients. 
»0 The overall spatial quality is fair. Recovered trick-play images 

are recognizable and are acceptable for tape search purposes. 

The following table summarizes trick-play quality 
achieved by the various inventive methods disclosed. 
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■ 


REAL-TIME TRICK 
PLAY STREAM 
GENERATION 


NON-REAL-TIME 
TRICK PLAY STREAM 
GENERATION 


FULL 
RESOLUTION 
TRICK PLAY 

MOl^ 


SP>\TIA1, OUALITY: 


SPATIAL QUALITY; 


poor to fair, only DC 
coefficients used. 

XPMPORAL OUALITY: 


poor to fair, DC & 
some AC coefficients 
used. 

TFMPORAL OUALITY: 


poor to acceptable, 
holding times 


poor to acceptable, 
constant S frame 
holding time. 


LOW 
RESOLUTION 
TRICK PLAY 
MOC^ 


SPATIAL OUALITY: 


SPATIAL OUALITY: 


poor to good, depends 
on material, 
patchwork of MBs 
used. 

TPMPORAL OUALITY: 


good to very good, 
depends on material, 

TPVTPORAL OUALITY: 


good, constant 3 
frame holding time. 


very good, constant 2 
frame holding time. 



In view of the constraints discussed previously, the highest trick-play 
quality may be achieved, in both real-time and pre-recorded 

5 material, by the use of lower-resolution trick-play data. However, 
the advanced television receiver/decoder must support the use of a 
low resolution mode. If full-resolution trick-play modes are utilized, 
the quality provided may be enhanced by manipulation of various 
parameters. For example, raising the effective bit-rate available for 

10 each trick-play speed, will allow an increase in resolution. However, 
a minimum bit-rate of approximately 2.0 Mbps is required. If the 
number of "Trick Play" speeds provided is reduced, for example to 
two in each direction, then the effective bit-rate for each remaining 
speed may be increased. The effective temporal resolution, or 

15 number of frame repeats, results from the trade-off between 

temporal and spatial resolution. Hence either parameter may be 
optimized depending on the desired application. 
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CLAIMS 

1 A method for generating an MPEG compatible digital image 
representative signal for recording to facilitate reproduction at more 
than one speed, said method comprising the steps of: 

a) receiving a digital image representative signal(09). 

b) temporally sub-sampling said digital image 
representative signal (09) at a rate related to a 

c) encoding said temporally sub-sampled signal (56. 
66. 76) to produce an MPEG compatible trick play signal (121, 131. 

9 141)* 

,0 ' d) encoding said digital image represenutive data 

signal (09) to produce a normal play MPEG compatible signal (101); 
s.gna. (O ) P ^^^^^^ ^^^^^ compatible 

„ signal (121. 131. 141) and said normal play MPEG comparible stgnal 
14 (101) to produce a record formatted bit stream (200); and. 
J 5 f) recording said record formatted bit stream (200). 

1 2 TTie method of claim 1, wherein said trick play MPEG 

2 compatible signal (121. 131. 141) comprises I frames. 

3 The method of claim 1. wherein said trick play MPEG 
compatible signal (121. 131. 141) is selected to occur '" '^^ "^^ 
formatted bit stream (200) such that when J"" "fj/f 
MPEG compatible bit stream (121. 131. 141) is reproduced a. sard 

5 trick play speed. 

4 The method of claim 2. wherein said 1 frames comprise at 
least a DC discrete cosine transform coefficient and an AC discrete 



1 
2 
3 
4 
5 
6 
7 
8 



1 1 
12 
13 



1 

2 
3 
4 



1 

2 



3 cosine transform coefficient. 



5 The method of claim 1, wherein said temporally sub- 
sampling occurs at a rate determined by a predetermined frame 



1 

2 

3 repeat rate. 
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1 6. The method of claim 2, wherein said step c) additionally 

2 includes; 

3 setting a DSM_trick_play_flag in a Packetized Elemental 

4 Stream layer of said MPEG compatible trick play signal (121, 131. 

5 141) to repeat said I frame. 

1 7. A method for deriving MPEG compatible digital image 

2 representative signals for record formatting to facilitate reproduction 

3 at more than one speed, said method comprising the steps of: 

4 a) receiving a digital image representative signal (09): 

5 b) temporally sub-sampling (55. 65. 75) said digital 

6 image representative signal (09) to generate a signal (56. 66, 76) 

7 having an frame rate related to a trick play speed; 

g c) encoding (100) and storing (560. 570. 580) said 

9 signal (56. 66. 76) having said frame rate related to said trick play 

10 speed to produce a trick play speed MPEG compatible signal (521. 

1 1 531, 541); 

12 d) encoding (100) and storing said digital image 

13 representative signal (09) to produce a normal play speed MPEG 

14 compatible signal (501); 

15 e) retrieving said stored trick play speed and normal 

16 play speed MPEG compatible signals ( 501, 521, 531. 541); 

17 f) selecting between said retrieved normal play speed 
1 8 MPEG compatible signal (501); and said retrieved trick play speed 

19 MPEG compatible signal (521. 531, 541); to format a bit stream (200) 

20 which facilitates reproduction at said normal play speed and said 

21 trick play speed; and, 

22 g) recording said formatted bit stream (200). 

1 8. The method of claim 7, wherein said trick play speed MPEG 

2 compatible signal (521, 531. 541) is selected to occur in said record 

3 formatted bit stream (200) such that when recorded said trick play 

4 speed MPEG compatible bit stream (521, 531, 541) is reproduced at 

5 said trick play speed. 
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9 The method of claim 8. wherein said selecting step « 
controlled responsive to formatting control signal (FMT CTRl^^^n..6 
from a recorder (210) recording said formatted bit stream (200). 

10 A method for deriving and recording MPEG compatible 
digital image representative signals to facilitate reproduction at more 
than one play speed, said method comprising the steps of: 

a) receiving a digital image representative signal (09). 

b) temporally sub-sampling (55. 65, 75) said digital 
„ image representative signal (09) to generate a signal having an frame 
7 rate related to a trick play speed (56. 66. 76); 

0 d) MPEG encoding (100) and stonng (560. 570. 580) 
said trick play speed specific signal (56. 66 76); to P'<>^^^^^^^^^ 
Play speed specific MPEG compatible signal (521. 531. 541). 
^ ^ e) encoding (100) and storing (550) said digital image 

_ representative signal (09) to produce an MPEG compatible signal 

13 (501) specific to a normal play speed; 

14 f) reading from storage said trick play speed specific 

15 MPEG compatible signal (521. 531. 541); 

g) reading from storage said normal play speed 

1 7 soccific MPEG compatible signal (501); and 

g h) seLting between said trick play speed specific 

MPEG compatible signal (521. 531. 541) and said normal play speed 
specific MPEG compatible signal (501) responsive to a user con^o 
21 command (REM CTRL) to output an MPEG compatible signal (202) for 

2 2 decoding (07) to facilitate viewing of said digital image 

23 representative signal at said normal play speed and said trick play 

24 speed. 
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